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ABSTRACT 


The present Department of the Navy Budget process does not appear to include a 
completely satisfactory method by which to estimate the costs associated with programs re- 
quiring advancement of technology beyond the current state-of-the-art (SOA). Navy budget 
analysts could benefit from an understanding of how cost estimating is accomplished in the 
defense industry. With these insights, Navy estimating methods might be enhanced by al- 
lowing for the validation of the various estimating methods and inclusion within the Navy 
estimation process as appropriate. This research examines the methods used by a major de- 
fense contractor in estimating the development costs associated with a specific state-of-the- 
art extension project. The study, conducted through the use of personal interviews and ex- 
amination of project cost data and information, determined that a combination of techniques 
was utilized within a formalized contractor-specific estimation process. These techniques 
included engineering estimates, reference to similar projects previously accomplished, and 


parametric modeling. 
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I. INTRODUCTION 


A. RESEARCH OBJECTIVE 

Development of new and improved weapon systems requires technology to expand its 
boundaries. The Department of the Navy is a prime requirer of these state-of-the-art (SOA) 
technological advancements. However, the budgetary impacts of SOA extensions are often 
greater than anticipated. The Navy budget development process does not appear to include 
an adequate methodology for determining what extending an existing weapon system 
"should cost". To realistically budget for SOA extensions, Department of the Navy budget 
analysts should better understand how cost estimating for SOA extension projects is cur- 
rently accomplished by the individual(s) responsible for advancing this technology, i.e. the 
defense industry. Navy budget analysts can use this knowledge to test the accuracy of the 
various cost estimating techniques. The purpose of this research was to gain greater insight 
into how one major corporation involved in Navy contracting forecasts or predicts the de- 


velopment costs associated with extending technology beyond current bounds. 


B. RESEARCH QUESTIONS 
Given the preceding objective, the following primary research question was posed: 
What methods were employed by a major Department of the Navy contractor to predict the 
development costs associated with an actual state-of-the-art (SOA) extension project? 
In addition, the following subsidiary research questions were considered relevant in 


thoroughly addressing the primary question: 


1. What were the expected costs, actual costs, and variances experienced during the 
development of the SOA extension project? 


2. What were the reasons for the variances experienced? 


3. Which development cost categories had the most effect on total project cost?, i.e. 
what were the "cost drivers”? 


4. What organizational design or structure was utilized during the SOA project; and 
how does that design affect cost management? 


C. SCOPE, LIMITATIONS, AND ASSUMPTIONS 
l. Scope 

The research focused on the actual cost estimation techniques utilized by a De- 
partment of the Navy contractor in the development of an SOA extension project. No effort 
was made to consider the justification or fairness of profit margins or return on invest- 
ments; rather emphasis was placed on determining the development costs associated with 
the SOA extension. In addition, the study did not attempt to construct or suggest any ana- 
lytical models for predictive or explanatory purposes. The basic effort was aimed at pro- 
ducing an accurate description of the quantitative and qualitative development costing 
methods employed by a particular defense contractor. There has been no prior research di- 
rectly addressing the primary research question. Consequently, the researcher relied upon 
personal interviews with key contractor personnel and the collection and examination of 
cost data at the contractor site. The results of this study should provide the reader with a 
complete understanding of the actual process involved and results achieved by a specific 
contractor on an actual SOA extension project. 

2. Limitations 

The study is limited to the observations and conclusions reached by the re- 
searcher on an actual SOA extension project conducted by a Department of the Navy con- 
tractor. An inherent limitation of this study is that direct transferability of the data, findings, 
and observations to other applications may be inappropriate. The actual SOA project stud- 
ied involved the design , construction, and operational testing of a single final product. 
Production costs are seen as incorporated in the development costs and are therefore not 


considered separately. 


3. Assumptions 
The presumption throughout this research effort is that the Department of the 
Navy contractor routinely employs a method of estimating development costs associated 
with SOA extension projects. No overly stringent assumptions were made concerning the 
readers in-depth knowledge of cost accounting, quantitative techniques for cost estimation, 


or the acquisition/contracting process within the Department of Defense. 


D. RESEARCH METHODOLOGY 

The research methodology utilized in this effort consisted of a case study involving the 
cost estimating processes employed by Lockheed Missiles and Space Company in devel- 
opment of the Control System Test Vehicle (CSTV) project. Personal interviews and doc- 
umentation provided by the Naval Plant Representative Officer (NAVPRO) , Lockheed 
Corporation, Sunnyvale, California, and the Lockheed Missiles and Space Company, pro- 
vided the primary input for the research effort. Additional background research consisted of 
examining the literature base thorough the Defense Technical Information Center (DTIC), 
the Defense Logistics Studies Information Exchange (DLSIE), and various publications 


and journals. 


E. ORGANIZATION OF THE REPORT 

Chapter II begins with a discussion of relevant definitions and concepts found in the 
literature; such as, what constitutes an extension of the current state-of-the-art; what items 
are categorized as development costs; and the differences between cost growth involving 
contract modifications and true cost overruns or variances. Chapter II continues with a brief 
discussion of the types of contracts used to support SOA extension projects. Generic cost 
estimation techniques and methods used by the Department of the Navy and the defense 
manufacturing industry are also considered. Chapter III introduces the Control System Test 


Vehicle (CSTV) itself and the cost estimating methods employed by LMSC in estimating 


the project's development costs. Chapter III also includes a description of the orga- 
nizational structure and working relationships formed within LMSC to complete the Con- 
trol System Test Vehicle (CSTV) project. Chapter IV presents the cost data collected. Vari- 
ances between predicted and actual costs are examined along with the causative factors in- 
volved. Cost categories that exhibited significant influence over total project cost are identi- 
fied and discussed at the end of Chapter IV. Chapter V summarizes the principle findings 
of the study, conclusions reached, and practical recommendations made. In addition, areas 


for further research are suggested. 
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IIl. BACKGROUND REVIEW AND THEORETICAL 
FRAMEWORK 


A. INTRODUCTION 

The purpose of this chapter is to provide the reader with an overall perspective of the 
environment in which the research was conducted. The chapter begins by examining and 
defining relevant concepts and terminology associated with technical advancements in state- 
of-the-art weapons systems. The important difference between cost deviations due to 
modifications in the basic contract and true cost overruns is then discussed. What follows 
1s a deseription of a key project management tool used within the contracting effort; the 
Contract Data Requirements List (DD Form 1423). A brief look at the major types of gov- 
ernment contracts awarded to state-of-the-art extension projects provides the reader with a 
general understanding of the incentives and policies under which a development contract is 
administered. The two basic approaches currently used by the Department of the Navy to 
esumate development costs are then introduced. Finally, the most widely accepted generic 


cost estimating techniques currently in use by the defense industry will be presented. 


B. DEFINITIONS AND CONCEPTS 
P. -of-the- X l 
Before any discussion, a definition of what constitutes a state-of-the-art (SOA) 
extension, must be established. Although, an exact definition of the term is difficult to ob- 
tain, synonymous definitions are found in the technical literature. James R. Bright and 


Milton E. F. Shoeman describe the process of technological innovation as follows: 


The process of technological innovation is a phase intended to embrace those activities 
by which technical knowledge is translated into a physical reality and is used on a scale 


having substantial societal impact . This definition includes more than the act of inven- 

tion. It includes initiation of the technical idea and acquisition of necessary knowledge, its 

transformation into useable hardware (or a process), 1ts introduction into society, and its 
diffusion and adoption to the point where its impact is significant. [ Ref. 1:p.48 ] 

Perhaps the most precise definition of "state-of-the-art" technology and techno- 

logical advancement encountered by the researcher was given by G.N. Dodson and C.A. 


Graves of General Research Corporation as follows: 


The state-of-the-art is the state of best implemented technology reflected in the most re- 
cently [applied] physical and performance characteristics. Our overall hypothesis con- 
cerning the advancement in the state-of-the-art is more precisely stated as the develop- 
mental design characteristics, (or related to the advance of these characteristics) repre- 
sented in relation to the best that has been previously implemented. [ Ref. 2:p.1 ] 

With the above ideas in mind, the researcher has adopted the following as an ac- 
ceptable definition of state-of-the-art extension: advancements in applied technology, 
through the unique combination of known and newly developed technological methods, to 
produce a previously nonexistent product. Work currently in progress on the Strategic De- 
fense Initiative (SDI) program provides numerous examples of advancing technology 
through state-of-the-art extensions. The Exoatmospheric Reentry Vehicle Interception Sys- 
tem (ERIS) program ! under development at Lockheed Corporation combines technologies 
developed during previous projects (propulsion, ordnance, structural) with current and yet 
undeveloped technology ( advanced avionics, engineering integration) to produce a cur- 
rently non existent defensive missile system. 

2. Development Cost 


As stated in Chapter One, the researcher has limited the scope of the study 


exclusively to considerations of development costs. It may now be helpful to define 


| The ERIS program is currently being developed for the U.S. Army Strategic 
Defense Command ( USASDO) at an estimated cost of $500M. The ERIS program will 
consist of a system of ground launched , non-nuclear missiles, whose purpose will be to 
intercept and destroy incoming ICBMs. 


precisely what costs are included within this category. Development efforts can be viewed 
(within the Department of Defense) as a subset of a broader category of activity generally 
referred to as Research, Development, Test, and Evaluation (RDT&E). According to the 
Department of the Navy Budget Guidance Manual, the overall function of RDT&E is to 
provide the capabilities needed to most effectively carry out the tasks required to success- 
fully complete the mission of the Navy [ Ref. 3 ]. Moreover, RDT&E programs should fo- 
cus on (1) determining what technologies are possible, and (2) applying what is technically 
possible, to develop workable solutions that satisfy mission requirements [ Ref. 3 ]. The 
costs associated with these objectives are categorized as RTD&E costs. For the purpose of 
this study development costs will refer to all costs that are associated with the design and 
testing of a new operational capability ? . 
3. Cost Growth vs, Cost Overrun 

In a subsequent chapter, frequent reference will be made to the terms ‘cost 
growth" and "cost overrun" (or variances). There is often confusion regarding these two 
concepts; and therefore a tendency to erroneously use the two terms interchangeably. Cost 
growth refers to change in current cost estimates over a previously established base figure. 
Therefore, changes in the total estimated cost of a program should correctly be called "cost 
growth". Contract modifications, which are simply changes to the original specifications 
delineated in the contractual agreement, are a major reason for cost growth in a contract. 
Cost overruns on the other hand, denotes the difference between actual cost experienced 
and the estimated cost delineated in the contract; i.e. the estimated costs remain unchanged. 


Bc 


? As will be seen in Chapter Three, the development costs associated with the CSTV 
project also includes the cost to produce the single operational test vehicle. 


IS 


4. Types of Government Contracts 

Although the underling purpose of this paper is not to explore the various cate- 
gories of contractual agreements that exist between government and private industry, it is 
useful for the reader to acquire a broad understanding of the major types of contracts that 
are utilized in support of state-of-the-art extension projects. In general, a contract can be 
seen as an offer and acceptance backed by legal considerations. The types of contracts 
normally used to support RDT&E efforts (and thus include state-of-the-art extension pro- 
jects) are discussed below. 

Cost contract- A cost contract calls for the government to pay all allowable costs 
involved in executing a given research project. The contractor receives no fee. This type of 
contract establishes an estimate of the total costs as defined in the contract for purposes of 
(1) obligating current funds, and (2) establishing a ceiling beyond which the contractor 
cannot proceed (except at his own risk) without prior approval. [ Ref. 5:p. 16-3 ] 

Cost-Sharing contract- Under a cost-sharing contract the contractor is reimbursed 
for an agreed portion of his allowable costs, not to exceed an established ceiling without 
fee. [ Ref. 5: p. 16-3 ] 

Cost-plus-fixed-fee contract- The cost plus a fixed fee contract is similar to the 
cost contract in that it provides for payment to the contractor of all allowable costs as de- 
fined in the contract, and establishes an estimate of the total cost. In addition, it provides 
for the payment of a fixed fee based primarily on the nature of work to be performed [ Ref 
5:p.16-3 ]. As will be seen in subsequent chapters, the CSTV project involved this specific 
type of contractual agreement. 

Cost-plus-incentive-fee contract- The cost plus an incentive fee contract is a cost 
reimbursement type agreement with provision for a fee which is adjustable by formula in 


accordance with the relationship which total allowable costs bear to target costs. Under this 


type of contract, there is negotiated initially a target cost, a target fee, a minimum and 
maximum fee, and a fee adjustment formula. Factors other than cost, including perfor- 
mance and progress, can also be used as a basis for contract incentive. [ Ref. 5:p. 16-3 ] 

Fixed-price-incentive contract- The fixed price incentive contract includes a pro- 
vision for the adjustment of profit and the establishment of the final contract price by a for- 
mula based on the relationship which final negotiated total cost bears to target costs. Under 
this type of agreement, target cost, profit, price ceiling, and a formula for establishing final 
profit and price are negotiated prior to execution. [ Ref. 5:p. 16-3] 

Firm-fixed-price contract- The firm fixed price contract provides for a price 
which is not subject to any adjustment by reason of the cost experience of the contractor in 
performance of the contract. This type of agreement, when appropriately applied, places 
maximum risk upon the contractor. Because the contractor assumes full responsibility, in 
the form of profit or loss for all costs under or over the firm fixed price, he has a maximum 
profit incentive for effective cost control and contract performance. The firm fixed price 
contract is suitable for use in procurements in which reasonably definitive design and/or 
performance specifications are known and fair and reasonable prices can be established at 
the outset. This type of contract is also suitable for level-of-effort work in which the con- 
tractor is compensated for expending his best effort at fulfilling program requirements. 
[Ref. 5:p. 16-3] 

5. Contract Data Requirements List (DD Form 1423) 

A presentation of the Contract Data Requirements List (CDRL) is appropriate 
here since it will be referred to extensively in subsequent chapters. In accordance with DOD 
instruction 5010.12, all government contracts which require data as a deliverable item must 
have these requirements delineated in a Contract Data Requirements List (CDRL). This 


documentation must state all data requirements that the contractor is to furnish. Some of the 
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data that is included are: technical status reports, cost and schedule reports, test results, 
technical manuals, design drafts and specification listings, and specific analyses. For many 
of these items, an approved Department of Defense identification number, known as a Data 


Item Description (DID), exists and must be cited. [ Ref. 6:p. 4-52 ] 


C. NAVY COST ESTIMATION METHODS 
Inherent in all Department of Defense management decisions is the obligation to pro- 
vide for the highest mission capability possible within the limits of available resources. The 


policy on overall resource allocation is stated in Department of Defense directive 5000.1: 


A cost effective balance must be achieved among acquisition costs, ownership costs...., 
and system effectiveness in terms of mission to be performed. [ Ref. 7:p. 1 ] 


Throughout the Department of the Navy, there is much importance placed on accu- 
rately estimating the probable development costs associated with all program acquisitions, 
including state-of-the-art extension projects. Cost estimating efforts are found in every 
phase of the Navy planning, programming, and budgeting cycle (PPBS) as well as phases 
within the acquisition process. However, emphasis on cost estimating efforts is particularly 
predominant in the planning phase of the PPBS. The accurate estimation of development 
cost is an essential prerequisite to realistic budgeting for weapon systems.[ Ref. 8:p.3 ] 

The development of cost estimates for a particular program is the responsibility of the 
Principal Developing Activity (PDA). At the same time, independent cost estimates are 
produced by the Director of Navy Program Planning (OP-90). Finally, the DOD Cost 
Analysis Improvement Group (CAIG) independently conducts a comprehensive review 
and evaluation of both estimates and provides this information to the Joint Requirements 
and Management Board (JRMB). The primary costing methodologies that are employed by 


these activities are (1) to work from detailed estimates of the cost of work packages to 
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derive the overall estimate, and/or (2) to start from the overall characteristics of the 
particular system and estimate the probable development costs by deductive reasoning. 
[ Ref. 8:p. 10 ] 

The detailed estimation approach is referred to as the Engineering or "bottom up” 
method. It involves breaking down the project into separate and identifiable segments of 
work. The breakdown is accomplished by means of a Work Breakdown Structure (WBS). 


The WBS 1s defined in Military Standard 881A as: 


... 2 produce-oriented family tree composed of hardware, services and data which results 
from project engineering efforts during the development and production of a defense ma- 
teriel item, and which completely defines the project/performance. A WBS displays and 
defines the product(s) to be developed or produced and relates the elements of work to be 
accomplished to each other and to the end product. [ Ref. 9] 


With the task elements of the project identified in terms of the work breakdown struc- 
ture, development costs are estimated using available historical cost data and totalled at each 
level. An overall developmental cost estimate consists of a summation of the individual de- 
velopment costs of each task element. [ Ref. 8:p. 8 ] 

The second major cost estimation technique used by the Department of the Navy con- 
sists of initially viewing the project at the macro level. Specific physical and/or performance 
characteristics sought; such as size, complexity, or performance level, are then identified. 
Finally, derived relationships known as Cost Estimating Relationships (CER's) are applied 
to the parameters of the project to develop a total development cost estimate. This method is 
known as Parametric or "Top Down" modeling. The method (or combination of methods) 
used by the Navy depends on various factors including the availability of relevant historical 


data and/or the complexity of the project being considered. [ Ref. 8:p.9 ] 


D. INDUSTRY-WIDE COST ESTIMATION METHODS 

In subsequent chapters, reference is made to cost estimation techniques and processes 
currently employed by Department of Defense contractors. The parametric and engineering 
models were discussed in the preceding section. These two models are extensively em- 
ployed within private industry as well. What follows is a discussion of two additional 
models that are used to estimate the development costs associated with state-of-the-art ex- 
tension projects. 

l. Associated Program 

The associated program method of cost estimation attempts to compare the total 

development costs of a current project with that of a technically representative program 
previously completed. If the two projects are not completely identical, an attempt is made to 
adjust the cost estimate to reflect similarities and differences between the two projects. Be- 
cause of the difficulty in obtaining a previous project that can satisfactorily represent the 
current project, the associated program method is considered the least accurate method of 
estimation. It is most often used to obtain initial "starting points" early in the conceptual 
phase of the acquisition process and when little or no historical cost data over a wide range 
of programs is readily available. [ Ref. 10:p. 2-70 | 

2. Similar-To Method 

The similar-to or analog method of cost estimation closely resembles the associ- 

ated program method but with an important difference. Rather than basing the estimate on 
the cost experience of a single representative program, a historical database consisting of 
actual tasks and their associated costs, is utilized to estimate the development cost of the 


current project [ Ref. 10:p. 2-70 ]. The following example helps clarify the point. 


The use of new structural material for aircraft often requires the development of special 
cutting and forming techniques with manufacturing labor requirements that differ signifi- 
cantly from those based on sample primarily aluminum airframes. Faced with this prob- 
lem when titanium was first considered for use in airframe manufacture, airframe 


18 


companies developed standard-hour values for titanium fabrication on the basis of shop 
experience in fabricating test parts and sections. [ Ref. 11 ] 


The cost estimate for fabricating titanium airframes was based on an analogous 
task previously experienced when titanium test parts and sections were fabricated. The 
analogous experience does not come from a single past project; as task experience gained 
from numerous projects constitute the historical cost data. In fact, the similarity of the end 
products 1s not always overly relevant; what 1s important are the similar tasks themselves. 

Table I compares the major cost estimation methods currently in use by the De- 
partment of the Navy and defense industry contractors. An important point to observe is 
that each of these methods have inherent advantages and disadvantages and are used either 
singularly or in combination depending on various considerations. Table II illustrates the 
results of the Booze, Allen and Hamilton study of prevalent cost estimation techniques as 
related to various phases of the acquisition process [ Ref. 12 ]. As can be seen, there is 
evidence to suggest that the cost estimating methods employed throughout industry are not 
restricted to a single method,but rather, are used in concert with one another to generate a 
development and production cost estimate. This concept will be seen in a later chapter when 


the Lockheed Missiles and Space Estimating System Description (ESD) is presented. 


E. CHAPTER SUMMARY 

This chapter outlined several concepts considered relevant in estimating the cost of 
state-of-the-art extension projects. A workable definition of state-of-the-art extension was 
established followed by a brief look at development programs and development costs. The 
important difference between cost growth and cost overruns (variances) was then pre- 
sented. Various categories of government contracts used in the Research, Test, Develop- 
ment, and Evaluation (RTD&E) effort was generally reviewed. Finally, a description of the 
most prevalent cost estimating methods utilized by the Department of the Navy and the De- 


fense Industry was presented. 
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TABLE I 


COMPARISON OF ESTIMATION METHODS * 





Methodology Characteristics Pluses Minuses 
ASSOCIATED Based on total actuals + Provides early - Not sensitive to 
PROGRAM from technically quick-look technical/program- 
representative matic details and 
program modified by + Traceable to differences 
economic & complexity historical 
factors costs - Can't provide cost 
element breakdown 
PARAMETRIC Derived from relation- + Can be applied - Not traceable to 
MODELING ships of cost to non when system actual history 
cost physical or characteristics 
performance are known - Can't easily 
validate 
+ Can identify algorithm 
cost drivers 
SIMILAR-TO Generated from + High degree of - Requires relatively 
MODELING historical cost data accuracy mature design 


using tasks or 
equipment similar 
to proposed/concept 
design 


+ Provides cost 
breakdown 
in WBS format 


+ Models use 


own contractors 
data base 


- Requires main- 
enance of 
good historical 
data base 


- Requires cost 
analysts with 
extensive company 
experience 


ENGINEERING Detailed functional & 


Or 
"BOTTOM UP" 
MODELING 


cost element estimates 


prepared at lowest 


practical level of task 
and design definitions 


+ High degree of 
accuracy 


+ Provides WBS 
element costs 
to lowest level 


+ Uses available 
historical cost 
data 


- Requires detailed 
design description 


- Time consuming 
and expensive to 
generate 


- Can't use as stand 
alone approach 


* Source: Information received during interview with LMSC Estimating Systems Manager, Mr. T. Castro on 24 
September 1987. 
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TABLE II 


INDUSTRY COST ESTIMATING METHODS AND MOST PREVALENT 
USE BY ACQUISITION PHASE 


EARLY LATE 
FSD EI PRODUCTION 


PARAMETRIC N/A 


ANALOGY 


ENGINEERING 





P=Primary Method(s) CE=Concept & Exploration Phase 
S=Secondary Method(s) D/E=Demonstration & Validation Phase 
N/À- Not Typically Used FSD=Full Scale Development Phase 
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III. HISTORICAL PERSPECTIVE, PROGRAM STRUCTURE 
AND COST ESTIMATION PROCESSES 


A. INTRODUCTION 

This chapter focuses on the specific state-of-the-art extension project chosen for the 
Study, i.e. the Control System Test Vehicle (CSTV) developed by Lockheed Missile and 
Space Company. It begins by describing what the Control System Test Vehicle is, what it 
was designed to accomplish, and what Lockheed Missiles and Space Company's contrac- 
tual responsibilities are. The following section of the chapter describes the overall Lockheed 
Missiles and Space Company's (LMSC) management structure, lines of communication 
and authority, and the specific project structure incorporated during the completion of the 
Control System Test Vehicle (CSTV) project. The final section of the chapter examines the 
processes and methods utilized by Lockheed Missiles and Space Company in estimating the 


development costs associated with the state-of-the-art extension project. 


B. SYSTEM DESCRIPTION 

Prior to the age of nuclear submarines, the general understanding of submarine 
hydrodynamics was such that even though maneuvering responses were not always as ex- 
pected, the inherent dangers were few because of the slower operating speeds then experi- 
enced. However, with fleet introduction of high performance nuclear powered attack sub- 
marines, in particular the USS Los Angeles (SSN-688), these operating speeds were 
greatly increased as well as the achievement of maneuvering techniques previously 
unattainable. Extrapolation of data from computer simulation and small-scale models to de- 
termine the effect of submarine design modifications on the performance, stability and con- 


trol were found to yield less than reliable results. The ability to safely and fully exploit the 
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high performance potential of modern high speed submarines required a full understanding 
of all factors influencing their maneuverability and ability to recover from casualty situa- 
tions. To this end the Navy desired an improved method whereby data pertaining to the 
control, stability, and maneuverability characteristics of the Los Angeles class attack sub- 
marine could be gathered, analyzed, and understood. 

On July 25, 1978, Lockheed Missiles and Space Company formally responded to the 
Navy's request for proposal (RFP number NO0024-78-R-5352S) by submitting a techni- 
cal, management, and cost proposal calling for the design, construction, and test of a con- 
trol system test vehicle and related support equipment. What Lockheed Missiles and Space 
Company proposed with the introduction of the Control System Test Vehicle (CSTV) was 
the ability to provide a more efficient and effective means of assessing maneuvering re- 
sponses to control inputs with an accuracy never before attained. The Lockheed Missiles 
and Space Company concept involved the development of a 1/12 scale model of the Los 
Angeles (SSN 688) class nuclear submarine. It also included the accessories and support 
equipment needed for its operation and maintenance. Government furnished control and 
Instrumentation components were to be contained within a pressure hull and fully integrated 
with all other vehicle systems to form an unmanned, free running, self-propelled and con- 
trolled vehicle. The Control System Test Vehicle (CSTV) was designed to operate in fresh 
or Salt water to depths of 300 to 1,200 feet. The project was considered by Lockheed Mis- 
siles and Space Company (LMSC) to be a technically risky endeavor. This was primarily 
due to the fact that much of the technology needed to successfully complete the project 
would have to be integrated in a way never before attempted. In addition, the integrated 
electronics and government furnished computer hardware would have to be fitted into the 
confined space of the pressure hull. Furthermore, advancements in technologies of the time 


were required in the areas of advanced hydrodynamics, propellers, structures, fabricating 
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materials, anticorosion and fouling, and submerged mechanical and electronic component 
integration. 

On October 10, 1978, LMSC submitted its "Best and Final Offer" proposal to the 
Navy program manager (Naval Sea Systems Command) with a cost plus fixed fee contract 
( CSTV contract NOQ0024-79-C-5356) valued at $1,799,385 being awarded to LMSC on 
December 29, 1978. 


C. CONTRACTOR PROJECT RESPONSIBILITIES 

The responsibilities and obligations of LMSC in regards to the CSTV project can be 
found by referring to the Statement of Work (SOW). The complete SOW is included as 
Appendix A. In broad terms, Lockheed Missiles and Space Company, Inc., was the prime 
contractor for the project and was directly responsible for accomplishing the following 


tasks: 
1. The design and construction of one (1) Control System Test Vehicle (CSTV) 
2. The design and construction of a set of support equipment 
3. The test and evaluation of the vehicle to ensure proper operation 
4 


. To perform system engineering activities aimed at coordinating and controlling de- 
velopment and ultimate project completion 


D. ORGANIZATIONAL STRUCTURE 

For the Control System Test Vehicle Project, Lockheed Missiles and Space Company 
(LMSC) drew upon the management success previously realized by the Lockheed Califor- 
nia Company located in Burbank, California (a wholly independent subsidiary of the parent 
company, The Lockheed Corporation) and it's highly successful "Skunk Works" project 
team approach. Project organizations structured along task-team lines are utilized by all 
Lockheed subsidiaries for many of their advanced development or state-of-the-art extension 
programs. The CSTV project was therefore a fitting candidate for this project team ap- 


proach. The decision to organize the project in this manner was based on an internal 
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assessment of management elements seen as vital to the success of the CSTV project. 


These management elements are presented as follows: 


TABLE Il 
MANAGEMENT ELEMENTS SEEN AS VITAL TO THE CSTV PROJECT 


Authorized Project Manager, reporting to a Company Officer, full authority for 
his Project 

Establish a small and highly competent project organization 

Use a flexible and low cost drawing system that expedites and controls changes 
Keep documentation to a minimum, meeting CDRL requirements 

Maintain continuous status of the project visible to management 

Establish an austere and fully compliant Quality Assurance Program 

Support and participate in post-delivery testing to maintain competence for 
follow-on support 

Earn customer trust with integrity and close cooperation and liaison 

Obtain customer understanding and concurrence at the start of the contract on 


methods and procedures for controlling government furnished equipment and 
:echnical data 


Limit project access and the number of support personnel required 





The utilization of a project team approach was beneficial in controlling previous SOA 
project costs. Lockheed Corporation believes that the small number of people within a pro- 
ject team significantly improves management coordination and control. Furthermore, cost 
control responsibilities within the project manager's organization greatly increases cost 


awareness and the desire for controlling costs. 
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1. Overall Corporate Structure 

The overall organizational structure of the Lockheed Corporation is depicted in 
Figure 1, which also shows the relationship between the Control System Test Vehicle pro- 
ject team and the other organizational elements of Lockheed. The Lockheed Missiles and 
Space Company, Inc. (LMSC) is headquartered in Sunnyvale, California. Within LMSC, 
the Research and Development Division (R&DD) is responsible for developing new and 
advanced products that take maximum advantage of the skills available within the corpora- 
tion. Ocean Systems is one of the major product lines in R&DD. The CSTV project man- 
agement team was placed directly within Ocean Systems. Within this structural arrange- 
ment, the CSTV project team was authorized, and in fact encouraged, to draw upon all 
three major functional organizations within LMSC ( Missile Systems, Research and Devel- 
opment, and Space Systems) for technical development support and expertise. 

2 Li Lo un authori 

As was previously stated, Figure 1 illustrates the placement of the CSTV project 
within the overall Lockheed organization. This arrangement provided direct access to top 
executives within the corporation. This directness in communications was confirmed in 
several personal interviews with LMSC managers involved with the CSTV project and was 
considered an important advantage during the entire project. Also of significance was the 
authority given to the project management team during development of the CSTV project. 
Again, this seemed to be a positive aspect of the management structure, as expressed by the 
individuals associated with the project. 

3. CSTV Project Team Structure 

The overriding concept of the CSTV project management effort, i.e. tight project 

controls, and full authority and responsibility of the project manager, is reflected in the 


project team organizational chart (Figure 2). The project team structure can be seen as a 
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LOCKHEED CORPORATION 





LOCKHEED LOCKHEED 

OCKHEE LOCKHEED MISSILES & SPACE COMPANY SA 
COMPANY (LMSC) COMPANY 
LOCKHEED R.A. Fuh President LOCKHEED 
ELECTRONICS D d or SHIPBUILDING 
COMPANY COMPANY 
LOCKHEED 

AIRCRAFT 

COMPANY 


MISSILE RESEARCH & DEVELOPMENT SPACE 
SO ENS DIVISION SYSTEMS 


DIVISION H.P. Kerfoot, VP. & G.M. DIVISION 


OCEAN 
SYSTEMS 
DIVISION 

J.G. Wenzel, V P. 


OCEAN OPERATIONS CSTY PROJECT OCEAN SYSTEMS 
LABORATORY TEAM ENGI NEERING 


R.A. Tyrell B.L. Coffey, W.F. Hill, Project Manager | 
Resident Director Manager J g K.L Krug, Director 





Figure 1. Placement of the CSTV Project Team within Lockheed 


Source: LMSC Management proposal no. D085091 of 25 July 1978 
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Reports to J.G. Wenzel, Vice President, Ocean Systems 


CSTV PROJECT (1000) SENIOR T EG ISNI 
W.F. Hill, Project Manager ADVISORY BOARD 


PROJECT SYSTEM SAFETY 


CONTROLS (1420) 
(1510, 1520) 


QUALITY ASSURANCE 
(1430) 


REMCO 


ELECTRICAL & NAYAL ARCH 4 SUPPORT 
ELECTRONIC ENGRG. MARINE ENGRG. EQUIPMENT 


(1130,1140,1150) (1110, 1120, (1200 ALL) 
1160, 1170) 


MANUFACTURING SYSTEMS TEST AND 
ENGINEERING EYALUATION 


(1410, 1440, 1450, (1300 ALL) 
1460, 1470, 1530) 


() = WBS number 


Figure 2. The CSTV Project structure and WBS task responsibilities 


Source: LMSC Management proposal no. D085091 of 25 July 1978 
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hardware-oriented organization keyed to the Work Breakdown Structure (WBS). Major 
tasks and hardware elements are grouped based on similarity of content. Composed of key 
Ocean Systems engineers and Naval Architects, the Senior Technical Advisory Board, 
provided an important service for the CSTV project team. The board met quarterly and at 
the request of the CSTV project manager; to provide an independent audit and evaluation of 
technical and management progress. REMCO Hydraulics of Willets, California, the prime 
subcontractor was officially considered part of the CSTV project team. As developer of the 
CSTV's pressure hull, it was felt that close coordination and control of this most critical 


subassembly would best be achieved in this manner. 


E. WORK BREAKDOWN STRUCTURE 

A CSTV Work Breakdown Structure was used to control task definitions, work as- 
signments, budget allocations, development costs, and the assignment of personnel re- 
sponsibilities. It was seen as the master guide for identifying and evaluating all activities 
within the CSTV project. What follows is a description of the major elements of the WBS 
along with corresponding task identification numbers. 

1100- MODEL ( the Control System Test Vehicle itself). This element included all the 
technical analysis, design, fabrication, and procurement effort required to design, build, 
and deliver the complete CSTV vehicle incorporating both contractor furnished equipment 
(CFE) and government furnished equipment (GFE) items. Also included was the effort re- 
quired to prepare and deliver the related disclosure information, drawings, and reports 
called for by the Contract Data Requirements List (CDRL). Subelements included: Hull 
and Appendages; Propulsion Subsystems; Power System; Control and Recording System; 
Navigation System; Auxiliary Systems; and Project Integration and Assembly. The work 
on the last task element was completed when the CSTV vehicle was ready for integrated 


systems tests. 
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1200- SUPPORT EQUIPMENT. This element included all technical analysis, design, 
fabrication, and procurement efforts required to design, build, and deliver the support 
equipment needed for the full operation of the CSTV System. The effort also included all 
documentation, reports, interface coordination, and liaison with the government and GFE 
suppliers. Subelements included: Control and Display System; Trailer; Sled; Dollies; Ship- 
ping Containers; Battery Chargers; Auxiliary Power System; and Miscellaneous Equip- 
ment. 

1300- TEST AND EVALUATION. This element covered all efforts involved in plan- 
ning and conducting required operational tests, including preparation of plans, actually 
conducting the test, the analysis of the results, and the preparation of the test reports as re- 
quired by the CDRL. Subelements included: Development, System, and Acceptance Tests. 

1400- SYSTEM ENGINEERING. Efforts under this element included those resources 
required to analyze and define the systems detailed requirements and the establishment and 
control of interfaces. It included defining and monitoring the integration of the vehicle, its 
support equipment, and all GFE components. Subelements included: Systems Engineering; 
System Safety; Quality Assurance, Reliability, and Maintainability Programs; Human En- 
gineering; and Mockups. 

1500- SYSTEMS MANAGEMENT. This element covered overall Project Manage- 
ment activities including technical and management direction, project controls, status 
evaluation, establishment of priorities, assignment of tasks, and other appropriate project 
management functions. Subelements included: Project Management; Configuration and 
Data Management; and Integrated Logistics Support. 

Task elements of the WBS are assigned a unique charge number identified as a work 
order/work authority (WO/WA); which is also an element of the LMSC cost accounting 


system. This charge number corresponds to the WBS series and is further identified within 
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the accounting system in terms of organization number, primary cost element, and resource 
code. All budgeting, scheduling, cost collection, and measurement of work status begins at 
the cost account level. Thus the WBS is also the primary management control mechanism 


for summarizing various project information for reporting or monitoring purposes. 


F. LMSC COST ESTIMATION SYSTEM 

What follows is a description of the generic process used by LMSC to estimate the de- 
velopment and production costs of all projects, including those associated with technologi- 
cal extensions of the state-of-the-art. This process is the actual series of steps employed by 
the CSTV project team in estimating the project's development costs.The specific estima- 
tion method(s) used in the CSTV project along with any significant deviations from the 
generic LMSC process are examined in the following section. 

The LMSC cost estimation process can be seen as a series of events, within phases, 
utilizing the various cost estimation techniques described in Chapter IT in an effort to pro- 
duce effective and creditable cost estimates. The primary considerations of the system are 


to: 
* Ensure that source data for estimates are current, accurate, and complete 
* Develop and maintain documentation in support of the estimates 
* Assign responsibilities for originating, reviewing, and approving the estimates 
* Utilize successful techniques for developing direct and indirect cost estimates 


1. Pre-Proposal Planning 
Figure 3 illustrates the key elements involved in the pre-proposal planning phase. 
The cost estimation process begins when a Program Manager is assigned to a new business 
activity. In the initial planning phase of a project, the Program Manager will form a com- 
bined Program Office Staff and a contract proposal team. This combined team is organized 
with clearly defined lines of communications and authority. The members of the proposal 


team consist of representatives from various functional divisions within the LMSC 
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Figure 3. Pre-Proposal Planning 


Source: LMSC Estimating System Description 
(ESD) Manual 
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organization who are believed to possess the specific skills and experience to plan the 
program and proposal efforts and to reliably estimate the projects' development and 
production costs. À teaming agreement ? is authorized if : 1) the capabilities required are 
not available within LMSC, or 2) the unique capabilities of LMSC personnel coupled with 
non-LMSC team members are highly complementary and afford the customer the best 
combination of capabilities required to achieve system performance and cost objectives. 
The CSTV project did not require this arrangement as all capabilities were considered 
available within LMSC. Requests for Proposals (RFP) are used to communicate the 
government's requirements for a new contract. Once received, the proposal team begins the 
next phase of effort. 
2 irements Review and Developmen 

This phase of the estimation process involves a comprehensive review and inter- 
pretation of the technical and management requirements delineated in the RFP. Figure 4 is 
provided to include a complete presentation of the numerous events associated with this 
phase of the process. Only the most essential concerns will be discussed. 

The first action, the Proposal Schedule, established the ground rules and future 
milestones that are to be incorporated in the overall cost proposal development and plan and 
schedule. An important objective of the proposal team is for cost proposals to identify the 
conditions and assumptions taken into consideration during 1ts preparation. This is the fo- 
cus of the RFP review process. Its intentis to single out those task elements or conditions 


that may impact highly on contract costs. The specific items of consideration include 


3 A teaming agreement is a legally binding written agreement with another company 
(outside LMSC) to jointly prepare or conduct marketing research or development efforts 
leading to the award of a new contract. 
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Figure 4. Proposal Requirements Review and Development 
Phase 


Source: LMSC Estimating System Description (ESD) Manual 


the: 1) review of contract deliverables, 2) examination of contract options,* 3) review of 
the terms and conditions, 4) review of any funding constraints, 5) review of the GFE list, 
6) review SOW, WBS and WBS dictionary ( task elements) and 7) review of CDRL 
requirements. Depending on project complexity, the degree to which technological methods 
will be required to be advanced, and the stringency or difficulty of the RFP requirements, 
additional levels of program scheduling and review may be conducted to define more 
detailed activities. If required, this phase would occur somewhat after (but in some cases 
parallel to) the review phase. As part of the program scheduling and planning task, a make- 
or-buy analysis is also conducted. Often this is a formal requirement stated in the RFP. The 
CSTV project however, did not require this documentation, presumably because it was 
considered a development project involving no follow-on production. 

One of the most important LMSC documents resulting from this phase is the 
Program Requirements List (PRL). This list contains all equipment, subcontracts, and 
software needed for pricing. Also often included with the PRL is descriptive data on all 
listed items. These are known as fact sheets, and are put together by engineering divisions 
and are used in “bottom-up” cost estimation.The final two steps within this phase of the 
estimation process involve issuing quoting instructions and defining pricing logic ground 
rules. These instructions are intended to produce consistency and prevent confusion during 
the next phase of the process; the proposal estimation process. 

3. Proposal Estimation 
This section describes the LMSC process for determining and developing valid 


cost rationale and estimation. Figures 5A include all of the major steps involved in this 


^ A contract option is a unilateral right in a contract by which, for a specific time, the 
government may elect to purchase additional supplies and services called for in the contract, 
or may elect to extend the terms of the contract. 
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phase. For the purposes of this paper, only those steps used by the CSTV project team and 
any other essential steps will be reviewed. Itis also worth mentioning here that a significant 
number of the steps involved in the LMSC estimation process are designed to satisfy the 
voluminous documentation requirements established by the Federal Acquisition Regula- 
tions (FAR) as opposed to generating pure estimations of development and production cost 
of a particular project. Documentation requirements of the Federal Acquisition Regulations 
are beyond the scope of this study. Therefore, the process involved will be described in 
terms of the steps whose objectives are solely that of estimating development and produc- 
tion costs. 

This phase in the process begins with a "kick-off" meeting where cost input 
scheduling, cost/price estimating procedures, and other administrative preliminaries are es- 
tablished by the program manager. Once these ground rules are established, the actual cost 
estimation work begins. 

The total cost estimate is composed of three major cost categories. These are: di- 
rect materials (and its corresponding overhead costs), direct labor (and its corresponding 
overhead costs), and other direct costs. The material costs are further broken down into 
three categories and involve the following actions: 

A. Subcontracted material and purchased services: This category includes project spe- 
cific parts, components, reworked items and test, and consulting services that 
are not to be manufactured or provided directly by Lockheed itself. The solicitation 
of various subcontractors is done via formal Lockheed RFP's or informal request 
for quotations (RFQ). Often there is insufficient time to obtain subcontractor cost 
data before the Lockheed proposal due date. Moreover, as is the case with state-of- 
the-art developments, subcontractor supplied materials and the corresponding price 


quotations are not available. In this case, Lockheed will prepare an in-house 
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estimate based on a quote for similar effort from another subcontractor (analogy or 
similar-to method), or purchase order history from a similar contract (bottom-up 
method). 

Standard commercial material and raw material: This category includes standard 
items that Lockheed normally fabricates, in whole or in part, and that are generally 
stocked in inventory. Raw material consists of material in a form that requires 
further processing. Supplier quotations are the preferred method of obtaining cost 
data however, often as is the case with state-of-the-art developments, in house 
estimation is again utilized. 

Intra-Lockheed work transfer (IWT): This category includes all material items that 
are fabricated by Lockheed itself. The raw materials for these items are carried in 
Lockheed inventory and hence are accounted for at actual costs. No material 
estimation is therefore needed. 


An important step prior to the forwarding of a final consolidated Bill of Material 


(BOM) is a process called Price Analysis. This auxiliary analysis is performed whenever 


(1) the total BOM amount exceeds $100,000 and/or (2) the rate of technological change of 


the project is believed to include significant uncertainty as to justify additional review and 


consideration of the reasonableness of the cost estimation. The specific actions taken in 


conducting a Price Analysis, including the following: 


* 


* 


Comparison of price quotation received 


Comparison of prior price quotations with current current quotations for the same or 
similar end items 


Employing parametric modeling techniques as "sanity" checks, i.e. do these prices 
appear reasonable? 


Comparing prices set forth in published price lists with discount or quantity buy op- 
portunities 


Comparison of proposed prices with independently developed Lockheed estimates 
(similar-to method) 
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The end product of the material cost estimation process described above is a 
consolidated Bill of Material (BOM) composed of the individual cost categories. Final ac- 
tion on the BOM involves the application of a material escalation rate depending on the an- 
ucipated length of the particular project. 

The next category of cost to be considered is that of direct labor. Figure 5B 
illustrates the steps involved in this phase of the process.The estimates associated with 
direct labor are prepared and expressed in labor hours traceable to the research, design, 
development, and production of a particular project. Labor hour requirements for 
completion of the work to be performed as outlined in the statement of work (SOW) or 
work breakdown structure (WBS) are estimated. There are two basic methods used to 
arrive at these labor hour estimates.They are : (1) the Similar-To or Analog method; or (2) 
the Engineering or "Bottom-Up" method. The Similar -To method is the preferred 
technique. It generally yields the most accurate estimate if properly executed. However, 
this method requires the availability of extensive historical data upon which to base the es- 
timate. The CSTV project along with many state-of-the-art extension projects may involve a 
significant number of tasks in which little or no similar data exists. Therefore, the 
Engineering method is generally used to estimate the labor costs associated with state-of- 
the-art extension projects. The CSTV project involved the utilization of both methods, 
however much of the labor estimation was based upon engineering estimates. Once direct 
labor hours are estimated, they are converted to dollar costs by means of applying (1) cost 
standards; and (if appropriate) (2) learning curve theory. The cost standards are based on 
relating development and production costs to the specific characteristics of the project (such 
as composition, weight, size, or duration) and applying Lockheed and/or industry-wide 
statistics as appropriate. In general, learning curve theory states that the amount of time 


required to complete a given task will diminish over time as workers gain added experience 
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performing a given task. LMSC obtains the specific rate of learning for each labor task by 
employing the least-squares method of curve fitting to a LMSC specific labor cost 
database. Although learning curves may be correctly applied to some projects involving 
state-of-the-art extensions, the CSTV project did not include the application of a learning 
curve rate. This was apparently due to the unique nature of the project and the fact that no 
follow-on production was required. 

The final category of cost estimated is that associated with travel, overtime 
premiums, and other direct costs not previously included. Typically these costs are arrived 
at by similar-to techniques or direct quotations from supplying sources. The final phase of 
the LMSC cost estimating system involves a process of extensive management reviewout- 
side the auspices of the project team. Post-estimation phase techniques and considerations 


are the subject of the following section. 


G. OTHER CONSIDERATIONS AND COST MODELING 

A series of management reviews are conducted prior to the final proposal and follow- 
ing the approved cost estimates developed during the proposal estimating process. LMSC 
utilizes a parametric cost model; the RCA PRICE (Hardware) model, and a combined anal- 
ogy/associated model; the Lockheed STAR model. Discussion in this section is not meant 
to imply that their use is reserved solely for the later phases of the estimation process. 
These methods are used to provide credibility or "sanity" checks throughout the entire ac- 
quisition process. This concept is illustrated in Figure 6. 
As can be seen, the "mix" of cost estimation techniques utilized by LMSC is time phased 
and correlated with the project managers view of cost uncertainty. What follows is a brief 
presentation of the two methods used by LMSC to provide for creditability checks 


throughout the cost estimation process. 
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i: w Í 
The parametric model known as the RCA PRICE (Hardware) model is a com- 
mercially available decision support system that is appropriate for estimating development 
and/or production costs at the subassembly or higher level. It is not appropriate for use 
with extremely small component parts. Input into the model is achieved by means of mi- 
crocomputer with direct access through data link (MODEM) to the RCA PRICE mainframe 


computer. Input parameters include: 


(1) Physical factors: component weight in two domains; (a) the active electronic weight 
and (b) the mechanical/structural weight. These two input variables are considered 
critical to the accuracy of the RCA PRICE output. Additional physical factors 
include the volume and/or density of the component. These two inputs are not as 
Critical to output accuracy. 


(2) Qualitative inputs: component empirical data in three variables; (a) the electronic 
complexity, (b) the structural/mechanical complexity, and (c) the engineering com- 
plexity. The critical variable for developmental projects is the engineering 
complexity input; which pertains to the scope of hardware development task and the 
skills of the project team. Values for these quantitative factors are provided in 
matrix form by the model. 


(3) Other inputs: These parameters incorporate idea such as design repetition, number 
of prototypes to be developed, end units to be produced, production learning 
curves, and economic factors for escalation/de-escalation of labor rates. 

Input variables are completed for all identifiable hardware items of the system 
and are entered into the RCA PRICE model. The cost output received by the model is bro- 
ken down into hardware development, production, and total costs. Input parameters are re- 
peated and "what if" analysis is easily accomplished and thus cost categories that signifi- 
cantly affect total costs, i.e. cost drivers, can be identified. Several limitations of the RCA 
PRICE model are apparent. The most obvious is that input parameters require that system 
characteristics be known a priori to the estimate. This severely limits its application to 


state-of-the-art extension projects in that many of the characteristics are unknown or at best 


subjective estimates themselves. The accuracy of the output is also highly dependent on the 
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input parameters. In addition, the actual algorithm employed is considered proprietary in 
nature and thus is not easily validated. One would expect to find significant skepticism 
associated with this method of estimation. However, interviews with LMSC cost estima- 
tors conducted by the researcher revealed surprising support and credibility for the model. 
2. Lockheed STAR Model 

The Lockheed STAR cost estimation model is a combination of the characteris- 
tics inherent in both the associated and the similar-to models. The primary focus in the 
STAR model is the extensive use of detailed databases drawn from a large number of past 
Lockheed projects over a twenty year period. Three separate and distinct databases are 
maintained and used to generate cost estimates. Depending on the project under considera- 
tion, the Flight Hardware, Ground Hardware, or Software databases are accessed to cost 
data pertaining to a particular element or task being costed. The estimation model matches 
the task and project of interest to functionally analogous programs and historical cost expe- 
riences found within the database. Each model (Flight Hardware, Ground Hardware, or 
Software) uses an algorithm tailored to its own specific application and reflects the way in 
which Lockheed does business. If required, all three application programs can be integrated 
to produce a total system cost estimate. The STAR model databases do not include ocean 
systems project experience and hence was not utilized in the CSTV project. However, it is 


included in the discussion due to its extensive current use by LMSC cost estimators. 


H. SUMMARY OF THE COST ESTIMATION PROCESS USED FOR THE 
CSTV PROJECT 


The CSTV project closely followed the LMSC estimating system process previously 
discussed. What is now discussed is the estimating methods utilized in developing the 
CSTV project cost proposal. The predominant method used by LMSC to estimate the de- 


velopment costs for the CSTV contract proposals was the engineering or bottom-up 
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technique. Past experience with costs associated with similar projects was relied upon 
when comparisons were considered appropriate > . Often, however, the advanced technical 
nature of the CSTV project prevented direct applications and various adjustments were 
made in an attempt to improve estimation validity. 

LMSC's cost proposal for the Cost-Plus Fixed Fee (CPFF) contract used in support of 
the CSTV acquisition contained the following explanations of the development costs in- 


cluded and how these costs were derived: 


1. Direct Materials: The material requirements for the project were directly estimated 
based upon the Work Breakdown Structure (WBS) and the Statement of Work 
(SOW) documents. The costs were based on vendor quotes, catalog prices, previous 
purchase order history, and bottom-up engineering estimates. Common minor mate- 
rials (standard or common productive parts having broad applicability and wide us- 
age) were applied to direct materials by a negotiated fixed government rate. This rate 
is a LMSC standard rate based on historical costs. 


2. Material Overhead: This cost element represented the administrative burden associ- 
ated with material and subcontract costs in two categories; material and procurement. 
Prenegotiated fixed rates based on historical costs were applied to each item. 


3. Direct Labor: The estimated direct labor hour requirements for the CSTV project were 
derived from engineering estimates made by project team members.Task require- 
ments were identified by analysis of the CSTV project WBS. Labor classifications 
were made as follows: Staff Engineers, Specialized Engineers, General Engineers, 
Inspection, Technical Publications, and Manufacturing Development. Hourly labor 
rates, provided by LMSC's company-wide pooled labor rate database system, are 
applied to each labor classification total. These rates were incrementally escalated by 
cost of living/merit increase factors expected to be incurred. 


4. Labor Overhead: Labor overhead costs are divided into two categories; development 
overhead and manufacturing overhead. Specific rates were applied to the labor hour 
estimates of each category. These rates are company-wide, pre-negotiated and based 
upon historical LMSC cost experience. 


5. Travel Costs: Direct travel costs are based on the number, destination, and duration 
of trips expected to be required to satisfy the requirements for testing and special 
material item procurement. LMSC standard costs, based on tourist class, round trip 
airfares and government approved per diem rates, were used. 


> These similar experiences in which the CSTV project team referred to included the 
Deep Submergence Rescue Vehicle (DSRV), the Summa Ocean Mining Barge, and the 
Deep Quest Research Submersible projects. 
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6. Other Direct Costs: This category included four classifications to include reproduc- 
tion, computer, facility capital cost of money, and overtime premium costs. Repro- 
duction included labor, materials, and vendor support costs associated with the pro- 
cess of reproducing technical drawings, project plans, and blueprints. The rate per 
hour applied to total labor hours was a negotiated amount based on historical LMSC 
reproduction experience. Computer Assisted Design and Manufacturing (CADAM) 
techniques were used to produce 160 technical drawings. Structural stress and mo- 
ments of inertia analysis was accomplished using a UNIVAC 1110 computer. The 
estimated computer time for these tasks were based on engineering estimates. The 
applied rates per computer hour were negotiated and based upon historical LMSC 
data processing cost experience. Facilities Capital Cost of Money (FCCM) was ap- 
plied to direct labor hour estimates at a negotiated rates for all categories of indirect 
costs. The CSTV project team felt that unscheduled overtime would undoubtably 
occur as development and test functions were consciously extended and completed 
beyond an eight hour shift. Overtime premium costs associated with this likelihood 
was composed of two parts; estimated overtime hours and a negotiated rate to be ap- 
plied to these hours. The primary basis for the estimated overtime hours was the 
LMSC experience in association with a similar past project combined with the esti- 
mates of project members. 


7. General and Administrative Expense: A G&A expense rate was applied to the CSTV 
project's total estimated labor hours. This negotiated rate was based on historical 
LMSC experience. 


I. CHAPTER SUMMARY 

This chapter began with a system description of the Control System Test Vehicle pro- 
ject. Contractor responsibilities and task requirements were outlined and discussed in terms 
of the Statement of Work and the Work Breakdown Structure documents. Various organi- 
zational aspects of Lockheed, LMSC, and the CSTV project were seen as assimilating 
management policies and the project team structure of technical undertakings successfully 
accomplished previously within the Lockheed Corporation. The complex and comprehen- 
sive LMSC cost estimating system and the techniques utilized were presented from the per- 
spective of generating valid and accurate cost estimates. The concept of time phased cost 
estimates and "sanity" checks provided the reader with insights into how various cost esti- 
mation techniques are applied throughout the development process. The final section of this 
chapter points out significant techniques and considerations used in the development of cost 


estimates for the CSTV project. 
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IV. PRESENTATION AND ANALYSIS OF THE DATA 


A. INTRODUCTION 

The preceding chapter introduced the process and techniques by which LMSC gener- 
ates cost estimates for its state-of-the-art extension projects. This process was followed by 
the CSTV project team with very few exceptions. The purpose of this chapter is to present 
and analyze the results of the LMSC cost estimation process of the CSTV project. First, the 
original CSTV project cost estimate and the results of the Best and Final Offer contracting 
process are compared. The rationale for the revised cost estimates is then presented. Next, 
a presentation and analysis of the predicted and the actual development costs experienced is 
conducted. This data is presented in chronological order as was experienced throughout the 
life of the project and includes justifications for cost growth and variances amounts. Con- 
clusions are then reached concerning the principle factor(s) which significantly impacted 


CSTY project costs. 


B. INITIAL CSTV PROJECT COST ESTIMATES 

Appendix B contains the Contract Pricing Proposal (DD form 633-4) dated July 21, 
1978. This document is the result of the initial cost estimation effort on the part of the 
CSTV project team. As part of normal contracting procedures, a negotiation process be- 
tween representatives of LMSC and the government followed submission of the Contract 
Pricing Proposal. What resulted was a "Best and Final Offer" (BAFO) proposal submitted 
by LMSC on October 20, 1978. Appendix C contains the Contract Pricing Proposal (DD 
Form 633-4 dated October 18, 1978) which was provided as an enclosure to the BAFO 


submission. As is evident upon comparison of these two documents, the total estimated 


47 


project cost was reduced by a substantial sum © . It is important to realize that some of the 
difference in estimated project cost was due to changes in contract requirements as re- 
quested by the government. The following summary shows how the total contract values 
changed as a result of this negotiation phase. 


PRIOR TO BEST AND FINAL OFFER: 


Initial estimated cost $ 2,276,503 
Fixed fee 191.862 
Total CPFF contract amount $ 2,468,365 


AFTER BEST AND FINAL OFFER: 


Initial estimated cost $ 1,659,545 

Fixed fee 114 
Total CPFF contract amount $ 1.774.402 
DIFFERENCE: $ (693,963) 
PERCENTAGE REDUCTION: (39.11%) 


There were several reasons for the reduction in estimated project costs; and are pro- 


vided as follows. 


l. Direct Materials 


Materials and subcontracted items were reduced by $313,050 as a result of : 


(a) A change in the prime subcontractor responsible for manufacturing the 
pressure hull. This change (from REMCO to Niles Engineering) resulted in 
a significant price reduction. 


(b) An increase in government furnished items (GFT) as a result of negotiated 
contract changes. 


6 The BAFO figures, as will be seen, were not the final negotiated amounts. They are 
presented here to provide a complete accounting of the changes in cost estimates which 
occurred throughout the project. 
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(c) Market research into sources of continued suppliers allowed for substitution 
of lower per unit cost components. 


2. Direct Labor 
As a result of reevaluating the engineering process, estimated engineering labor 
requirements were reduced by 2,470 hours. Small economies and design improvements 
were effected to reduce the projected number of assembly drawings required. An additional 
833 labor hours of test engineering efforts were eliminated for being"non-essential" in na- 
ture. Total direct labor dollars were reduced from $ 524,743 to $434,850; a difference of 
$89,893. 
3 nuf rin 
Manufacturing labor estimates were reduced by 4,673 hours. This was due to 
numerous changes in the basic development contract. Because these changes were brought 
on by the government, for the purpose of this study , they are considered "contract reduc- 
tions " and should not be treated as true cost variances or overruns. 
4. Computer Time and Reproduction 
The expected project computer time was reduced by 992 hours; from $63,414 to 
$33,654, a difference of $29,760. This was a direct result of the revaluation of the engi- 
neering process and corresponding reduction in the number of projected assembly draw- 
ings required. The expected reproduction expense was also reduced by $1,220 for the 


above cited reasons. 


C. CSTV PROJECT COST GROWTH AND VARIANCE ANALYSIS 

As a result of further contract negotiations after LMSC submitted it's BAFO, an esti- 
mated project cost of $1,682,819, and a fee or profit of $116,566 was agreed upon. A 
CPFF contract was signed on 29 December 1978 and work on the CSTV project began 


soon afterwards. 
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Despite the thorough pre-contract award estimation efforts expended, a myriad of 
technical and scheduling difficulties were experienced throughout the development of the 
CSTV project. The initial estimated cost figure of $1,682,819 and the scheduled project 
completion time was formally changed a total of six times between August 24, 1979 and 
February 12, 1981. A combination of contract modifications and cost variances resulted in 
a final actual project cost of $3,979,838. The final completion date of the CSTV project 
was March 30, 1982; more than two years beyond the initial January 1980 target comple- 
tion date. All cost figures are nominal amounts measured in then-year dollars. This along 
with subsequent data was obtained by examination of LMSC documentation generated as a 
consequence of actual cost growth and overruns experienced throughout the life of the 
project. The negotiation/approval process associated with Cost Plus Fixed Fee (CPFF) 
contractual agreements, necessitated the format of the reports from which the researcher 
compiled the data. Accumulated costs are presented to the government for acceptance. The 
basis for these changes in estimated costs are justified by contract modifications and ex- 
pected cost variances (expected because LMSC cannot incur additional costs pnor to gov- 
ernment review and approval)’ . Thus a periodic negotiation process ensues between the 
government and LMSC with allowable costs resulting. 

Much of the data reviewed provided information on both costs and fee (profit). The 
objectives of the study limits its relevance to analysis of cost data exclusively. The fol- 
lowing table provides a chronological summary of the cost estimation changes actually ex- 


perienced throughout the life of the CSTV project. 


7 Actual cumulative costs form the basis for the total estimated project cost and is 
compared to the cumulative approved costs to determine the expected variance amount. See 
Appendix D. 
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TABLE III 


FINAL NEGOTIATED CONTRACT GROWTH & VARIANCE RESULTS 


DATE 


20 Dec 78 
24 Aug 79 
12 Nov 79 
20 Feb 80 
24 Jun 80 
29 Aug 80 
10 Dec 80 
12 Feb 81 
30 Mar 87 


ACTION 


Start Date 
Analysis #1 
Analysis #2 
Analysis #3 
Analysis #4 
Analysis #5 
Modification* 
Analysis #6 
Completion 


MODIFICATION 


N/A 
82 025 
192,986 


128,876 
239,817 
0 
75,000 


0 
$719,604 


VARIANCE 


N/A 
524,798 
3759973 
345,014 
209,970 

90,510 
0 
31,350 


Simone ails 


TOTAL 


007 725 
568,759 
473,890 
449,787 
90,510 
75,000 
21730 


$2,297,019 





* On this date, it was agreed upon to modify the contract by decreasing the fee by 


$75,000 and increasing the cost by the same amount. 


Detailed data (by the above actual reporting dates) conceming CSTV project cost 


growth and variance is provided in Appendix D and support all the data presented in the 


following pages. Reasons for these difference are worthy of examination so that insights 


into the risks, uncertainties, and technical difficulties associated with this particular state-of- 


the-art extension project may be made. Contract modifications as requested by the govern- 


ment will be identified but not be discussed in length as these estimated costs were devel- 


oped by means of the LMSC cost estimation process and models previously discussed. In- 


sights into the reasons for the cost variances will be the overriding objective.The primary 


rationale for each revisions of the cost estimate will now be discussed in the order in which 


they occurred throughout the life of the CSTV project. 
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1. CSTV Cost Growth and Variance Analysis Number One 
The first formal change in the CSTV cost estimate occurred on August 24, 1979. 


The cost revision as of this date is summarized as follows: 


Cost 
Basic Contract $1,682,819 
Contract Modifications 82,925 
Variance or Cost Overrun 524,798 
Revised Estimated Cost $2,290,542 
Estimated Project Completion Date May 1980 


The contract was modified at the governments request to include 1) the design 
and manufacture of a special boom assembly , 2) the procurement/design of special filters, 
and 3) an increase in the accuracy of a specific subsystem. The primary reason for the vari- 
ance experienced as of this date was directly contributable to the complexity and 
sophistication of the CSTV itself. In 1977, LMSC was awarded a preliminary design con- 
tract. Design studies then began in anticipation of a Navy RFP. Because of the extensive 
preliminary design work completed prior to the RFP, it was the opinion of LMSC that very 
little preliminary design would be required at the time that the contract would be let. As a 
consequence, the LMSC Best and Final Offer reflected only a minor amount of anticipated 
preliminary design costs. However, it soon became apparent that Navy specifications, es- 
pecially those dealing with the tail section design and other technical matters, were not be- 
ing precisely meet with the design efforts to date. Further preliminary design work was 
needed. Further problems surfaced once actual developmental work began. The complexity 
of the project required additional drawings ( from 136 to 321) and engineering effort in 
order to adequately define the product. In addition, a number of procurement difficulties 
were experienced. The most significant being the rejection of unsatisfactory hull material 


from Kaiser Aluminum. Reprocurement of this material amounted to project delays and 
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fruitless efforts at obtaining an alternate source of supply. Finally, significant quantities of 
highly specialized precision components, required by the CSTV, were fabricated by LMSC 
when they proved to be unavailable "off-the-shelf" as had been previously expected. In 
summary, the following factors contributed to the cost variance at this stage of project de- 


velopment: 


(a) Substantially more preliminary design effort than anticipated. 
(b) Significant procurement difficulties experienced. 
(c) Off-the-shelf components not available to the extent planned. 


2 V w Varian nalysi w 
The second formal change in the CSTV cost estimate occurred on November 12, 


1979. The cost revision as of this date is summarized as follows: 


Cost 
Allowable Costs to Date $2,341,255 
Contract Modifications 192,986 
Variance or Cost Overrun 375773 
Revised Estimated Cost $2.910.014 
Estimated Project Completion Date May 1980 


The contract was modified at the governments request to include 1) further ef- 
forts to design test and manufacture an additional tail section for the CSTV and 2) an up- 
graded electronics package. Since the first cost growth and variance analysis, a number of 
technical and schedule problems developed which directly impacted the estimated project 
completion costs as seen above. The first problem had to do with CSTV hull fabrication 
difficulties experienced at Niles Machine Inc., the prime subcontractor. The time to com- 
plete welding of inserts, closures, and foundations was greater than anticipated. Significant 
numbers of LMSC technicians were sent to the subcontractor site in an attempt to rectify the 
holdups. In the end it became necessary to transfer entire sections of the hull to LMSC for 


completion of welding work. Problems with the control electronics and procurement of a 
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critical ballast pump assembly was the second difficulty encountered by LMSC at this time. 
Electrical wiring changes were necessary after initial hull sections were received and found 
to be more densely packed than anticipated. Finally the overall complexity and cumulative 
design changes thus far in the project resulted in additional quality assurance, rework, and 
continued engineering design efforts. In summary, the following factors contributed di- 


rectly to the continued variances in development costs: 
(a) Hull fabrication difficulties. 
(b) Procurement difficulties experienced with major subassemblies. 
(c) Technical problems associated with wiring configuration greater than anticipated. 
(d) Additional design work Q/A, and rework beyond expectations. 


3. CSTV Vari ina rada 
The third formal change in the CSTV cost estimate occurred on February 29, 


1980. The cost revision as of this date is summarized as follows: 


Cost 
Allowable Costs to Date $2, 859,301 
Contract Modifications 128,876 
Variance or Cost Overrun 45.014 
Revised Estimated Cost $3,333,191 
Estimated Project Completion Date September 1980 


No additional contract modifications were identified as of this reporting date. 
However, a significant cost variance was experienced. Additionally, the CSTV project in- 
curred its first estimated program schedule extension. LMSC was experiencing several 
unanticipated procurement and subcontractor performance difficulties. Delays in manufac- 
turing critical path items, such as the sail and tail subassemblies, were traced to the lack of 
required component parts. These events not only resulted in project delays but caused 
LMSC to expend considerable efforts in resolving these problems. Cost overruns not di- 


rectly linked to project delays due to lack of components, primarily included continued 
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difficulues with hull fabrication and unexpected hardware integration problems between 
off-the-shelf components (also of which required custom-made connector parts). This 
Situation required the addition of more engineering and administrative man-hours than 
expected. Planned expenditures for material costs were not significant affected. A summary 


of the primary reasons for the increased development costs estimate include: 


(a) Delays associated with deliveries of component parts; some of which directly affect 
tasks on the critical path. 


(b) Difficulties with connections between components 
(c) Continued problems with hull fabrication 


4. CSTV Cost Growth and Variance Analysis Number Four 
The fourth formal change in the CSTV cost estimate occurred on June 24, 1980. 


The cost revision as of this date is summarized as follows: 


Cost 
Allowable Costs to Date $3,333,191 
Contract Modifications 239,817 
Variance or Cost Overrun 209,970 
Revised Estimated Cost $3.782.978 
Estimated Project Completion Date February 1981 


The contract was further modified at the government's request to include 1) a 
second tail assembly, and 2) a modification to the existing installed tape recording system. 
The cost variances experienced at this time primarily resulted from continuing problems 
remaining as of the last report date. These difficulties were expected to be resolved rela- 
tively quickly. As it turned out however, LMSC was unable to accomplish this without uti- 
lizing significant additional resources. The cost overrun was primarily attributed to contin- 
uous subcontractor difficulties which required additional LMSC personnel being sent to the 
subcontractor's sites in an attempt to rectify component design and manufacturing prob- 


lems. 


5. CSTV Variance Analysis Number Five 
The fifth formal change in the CSTV cost estimate occurred on August 29,1980. 


The cost revision as of this date is summarized as follows: 


Cost 
Allowable Costs to Date $3,782,978 
Variance or Cost Overrun 90,510 
Revised Estimated Cost $3,873,488 
Estimated Project Completion Date January 1981 


No additional requests for contract modifications was reported at the time of this 
cost analysis report. In fact, most of the problems experienced earlier seemed to be re- 
solved. The variance, as verified above, was the smallest anticipated thus far in the CSTV 
project life. However, a relatively new technical difficulty surfaced which caused the con- 
tinued high usage of certain engineering and manufacturing personnel. This involved re- 
pair, modification, and isolated testing of key components of the CSTV control mecha- 
nism. As will be seen, this problem was not completely resolved prior to the submission of 
the final variance report. 

6. V_Vari Lysi mber 
The final formal change in the CSTV cost estimate occurred on February 


12,1981. The cost revision as of this date is summarized as follows: 








Cost 
Allowable Costs to Date $3,948,488 * 
Variance or Cost Overrun 31,350 
Revised Estimated Cost $3,979,838 (Final Cost) 
Actual Project Completion Date March 1981 


* On 10 Dec. 80, it was agreed upon to further modify the contract by increasing 
allowable project costs by $75,000. 


No further government induced contract modifications were experienced through 


the completion of the CSTV project. The anticipated variance was attributed to remaining 
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technical problems associated with the CSTV control system. In addition, the costs for the 


second tail assembly and computer graphics time for final drawings were greater than ex- 


pected. 


The objective of the preceding paragraphs was to present the estimated and actual cost 
data in a format that would be easily comprehended by the reader. Furthermore, the reasons 
for the increase in project costs were provided so as to identify the various factors impact- 
ing on cost. As 1s apparent by observing the differences between the beginning cost esti- 
mate and the actual project cost results, these differences are significant. Even with a pro- 
cess as complete and exhaustively detailed as the LMSC approach, the process of cost es- 
timation in this particular case was an exceedingly challenging one. 

In the opinion of the researcher, the value of a model such as the LMSC estimating 
system stems not only from its capacity to accurately predict future costs but also in its 
ability to enhance the identification of factors influencing it's outcome. It was shown that 
the total difference is a combination of the CSTV project's cost growth and cost variance or 
overruns. It would appear that several factors throughout the development of the CSTV 
project contributed significantly to increases in actual cost. The degree of technological ex- 
tension required would be expected to directly affect development cost by increasing un- 
certainty associated with various task accomplishment. The CSTV project costs appeared to 
be adversely affected by this task uncertainty. In fact, by observing the data contained in 
Table III, it is interesting to note that variance amounts appear to be greatest in the initial 
stages of project development. Moreover, these amounts decrease with the life of the pro- 
ject and presumably with a reduction in task uncertainty. It was seen that extensive 
difficulties were initially experienced and that extended preliminary design work was 
required. The uncertainty factor seemed to shift away from LMSC personnel and more to- 


wards subcontractors. As was observed, the major reason for cost overruns in the later 
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stages of development appeared to be design and manufacturing difficulty with subcon- 
tractors. Numerous LMSC technical visits were conducted to assist subcontractors in suc- 
cessfully achieving key project tasks. Also apparent from the data included in Table III is 
the observation that contract modifications occurred more towards the beginning and mid- 
dle phases of the project than at the end. This may be a function of product definition and 
clarity as more tangible results are realized. To the extent that contract modifications affect 
variance amounts is less conclusive from observation of the data. A final factor that ap- 
peared to adversely affect development costs in this specific case is that of material delays 
of singularly less critical off-the-shelf components and their successful integration with 
other component parts. These difficulties caused severe delays in the project, unproductive 
efforts to find alternative sources of supplies, and excessive utilization of expensive engi- 


neering labor in redesign efforts. 


D. CHAPTER SUMMARY 

This chapter presented and analyzed the actual and estimated development costs 
associated with the CSTV project. Data were presented in chronological order as they were 
experienced throughout the life of the project. Modifications to the contract, as requested by 
the government, were identified. However primary attention was paid to the cost overruns. 
Reasons were then given for the resulting cost variances. Finally, several conclusions con- 


cerning factors which appeared to significantly affect development costs where reached. 
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V. PRINCIPLE FINDINGS, CONCLUSIONS, AND 
RECOMMENDATIONS 


A. PRINCIPLE FINDINGS AND CONCLUSIONS 

The primary objective of this study was to determine what methods were used in esti- 
mating the development costs associated with an actual state-of-the-art extension project. 
Lockheed Missiles and Space Company, Inc. and the development of the CSTV project 
provided the basis for this inquiry. The principle findings and conclusions were derived 
from personal interviews with key LMSC management personnel and relevant documenta- 
tion collected and analyzed at LMSC and the Lockheed Naval Plant Representative Office; 
both located in Sunnyvale, California. These findings and conclusions are presented and 
discussed as follows. 

A relatively complex and comprehensive process known as the L Estimating 
System, was utilized to predict the development cost of the CSTV project. This process 
consists of a series of events within phases. Along with producing creditable cost esti- 
mates, the estimating process was used to assign task assignments to accountable person- 
nel, and to thoroughly document cost estimates for internal and government audit purposes. 

A combination of cost estimating techniques were employed within the guidelines of 
he LMSC Estimating em to predict the development cost of the CSTV project. Al- 
though the Engineering or "Bottom-Up" cost estimating technique was predominantly uti- 
lized on the CSTV project, the associated program method was relied upon to a lesser ex- 
tent when program comparisons were considered appropriate. Parametric techniques were 
used in the final cost estimation review phase to provide "sanity" checks. They were not 


relied upon in the primary stages of generating cost estimates. 
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The CSTV project was organized around a project team approach . Numerous 
managerial requirements, including tight project control, and the establishment of a single 
responsible project manager, were accomplished in this manner. This project team organi- 
zational structure was based upon the successes experienced by several Lockheed compa- 
nies involved with similar SOA extension projects previously undertaken. 

Despite the elaborate nature of the LM cost estimation proce emploved, sig- 
nificant cost growth and overruns (variances) resulted. The final CSTV project cost in- 
creased dramatically over the initial estimated costs. This led the researcher to conclude that 
the process of cost estimation utilized by LMSC to estimate the CSTV project's costs was 


complicated by the degree of technological extension required. 


Task uncertainties associated with the development of the CSTV project appeared to be 
the major contributor to the cost overruns experienced. Although significant preliminary 


design work was conducted prior to the award of the CSTV contract, numerous additional 
engineering labor hours were incurred as project engineers attempted to adequately define 
the CSTV. These design efforts appeared to lessen in the later stages of project develop- 


ment. 


B. RECOMMENDATIONS 

1. Government Officials responsible for SOA extension projects should continue to 
thoroughly review cost estimates submitted by contractors. Emphasis and attention should 
be placed on whether, or to what extent, the contractor fully understands the requirements 
of the project. 

2. Continue to insure that contract documentation such as the SOW, CDRL, and 
WBS for state-of-the-art extension projects is clearly defined and understood by all con- 
cerned parties. This may help to minimize the possibility of project misunderstandings and 


underestimation of the costs associated with project development. 
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3. Move towards more fixed price contracts that place greater risk upon the contrac- 
tor. SOA extension projects appear to be filled with numerous uncertainties caused 
primarily by the degree to which extended technology is required. The process of estimat- 
ing expected costs that adequately accounts for project uncertainties was not successfully 
realized in this particular case and resulted in the government incurring significantly more 
costs than anticipated. This also may be the experience with cost estimation process em- 


ployed in other projects. 


C. RECOMMENDATIONS FOR FURTHER RESEARCH 

If total contract cost growth is used as a measure of effectiveness, it is apparent that the 
process utilized in estimating the final project cost was not, for various reasons, com- 
pletely successful in the case of the CSTV project. This study involved one specific case 
and cannot reasonably be expected to fully support any broad based theoretical conclusions 
on its own. It is possible but unlikely that the cost growth and overruns are unique to this 
particular case. Therefore, it is recommended that further case studies involving SOA ex- 
tension projects be conducted to determine if similar difficulties are encountered. The focus 
of these additional case studies should be to provide further identification of the factors af- 
fecting cost estimates and the means by which the degree of required technological exten- 
sion is incorporated into the cost estimation process. Greater insights into the process in- 
volved would assist Navy budget analysts and program managers in more accurately de- 


termining cost requirements and impacts of SOA extension projects. 


D. FINAL SUMMARY 
As a final summary of the information presented and discussed in this report, the pn- 


mary and subsidiary research questions will be reinstated and briefly answered. 
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2: 


Primary Research Question: 

What methods were employed by a major Department of the Navy contractor to predict 
the development costs associated with an actual state-of-the-art (SOA) extension 
project? 

Answer: A combination of the engineering ( primary technique), associated program, 
and parametric (used by LMSC cost estimation reviewing authorities) techniques 
within the framework of the LMSC Estimating System. 

Subsidiary Research Questions: 

What were the expected costs, actual costs, and variances experienced during the 
development of the SOA extension project? 

Answer: Details on costs and variances are provided in Chapter IV and in Appendix D. 
Significant cost variances and contract growth resulted. 

What were the reasons for the variances experienced? 

Answer: The primary reason stems from the costs associated with the engineering re 
design efforts required as a consequence of the uncertainty associated with the 
advanced and complex technological nature of the CSTV project. Additionally, an 
incomplete understanding of contract requirements lead to unexpected cost increases. 
Which development cost categories had the most effect on total project cost?, i.e. what 
were the ‘cost drivers"? 

Answer: In the early stages of the project, the engineering labor cost to adequately 
define the CSTV product was the cost category most affecting the project cost. In the 


later stages, costs were associated with subcontractor performance. 
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What organizational design or structure was utilized during the SOA project; and how 
does that design affect cost management? 

Answer: Á project team approach was utilized; the design of which was adapted from 
successful SOA extension projects previously undertaken by other organizations 
within the Corporation. Concern over project costs received a greater degree of 
management attention than would have occurred if more traditional organizational 


approaches had been adopted. 
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APPENDIX A 
CSTV STATEMENT OF WORK 


The Statement of Work describes the tasks to be preformed by LMSC in the design, con- 
struction, and test of the Control System Test Vehicle (CSTV) and support equipment. 
This information was reproduced from the LMSC management proposal for the CSTV 
project submitted to NAVSEA on 25 July 1978. 


SCOPE 

The contractor shall make maximum use of all data developed and documented under Con- 
tract N0004-78-C-5309. All work to be preformed under this contract is generally de- 
scribed below: 


ITEM 

0001- Design, construct, test and deliver one (1) Control System Test Vehicle and one (1) 
set of support equipment as defined herein. 

0002- Provide interim repair parts, supplies and services as ordered by the Administrative 
Contracting Officer (ACO) and authorized by contract modification. 


0003- Prepare and deliver as defined on the Contract Data Requirements List. 
PERIOD OF PERFORMANCE 
The contractor's effort described in this Statement of Work shall be accomplished during 


the periods specified below: 


ITEM 


0001- From contract award through the twelfth (12) month after contract award. 
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0002- As specified in the authorizing contract modification. 


0003- From contract award through the fourteenth (14) month after contract award. 


APPLICABLE DOCUMENTS 


The following documents of the issues and date specified form a part of this SOW to the 


extent specified herein. 


MILITARY 
NO-794-78-003 April 1978 "Control System Test Vehicle 
and Equipment Specification" 
Exhibit Á to 10 May 1978 ASCOP Test Vehicle Contract 
N00024-78-PR-31018 Data Requirements 
List (DD Form 1423) 
TASK DESCRIPTION 


This section defined the tasks to be performed by the contractor and the responsibility 
interfaces between the contractor, NAVSEA, and other participating activities. Paragraph 
numbers herein are directly relatable to the Work Breakdown Structure (WBS) included in 
the Control System Test Vehicle Specification, NO-794-78-003 and the contractor's Man- 
agement Proposal. 

Model Vehicle. Design and construct one Test Vehicle in accordance with the requirements 
of Specification NO-794-78-003 and Appendixes 1 through 5 thereof. 

* Perform Design Studies, Design Analyses, and Producibility Studies to support 


the detail design. 
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* Integrate GFE into the CSTV design, with specific emphasis on the Inertial 
Measurement Unit (IMU) and the Control and Recording System (CRS). 

* Generate detail design drawings. 

* Prepare a design Analysis Report (CDRL Sequence A004). 

* Prepare Critical Design Review Data (CDRL Sequence A003), and support the 
Critical Design Review. 

° Support internal design reviews. 

* Provide technical direction to subcontractors. 

e Manufacture, inspect, assemble, and test the Test Vehicle. 


* Prepare "As-built" drawings (CDRL Sequence AOOD). 


Support Equipment. Design and construct one set of Support Equipment consisting of the 
items listed below in accordance with the requirements of Specification NO-794-78-003, 
Appendix 6. 

° One control and display system. 

¢ Twelve (12) dollies. 

e Shipping containers as required. 

e One (1) external power supply. 


¢ Miscellaneous slings, umbilicals, etc. 


In support of this requirement, the contractor shall: 
* Perform design studies and analysis to support the detail design. 
* Generate detail design drawings. 
* Prepare Critical Design Review Data (CDRL Sequence A004), and support the 


Critical Design Review. 
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Support internal design reviews. 

Provide technical direction to suppliers of purchased equipment. 

Manufacture, inspect, assemble, and test the support equipment. 

Provide interface information as required to facilitate Government modification of 


the GFE trailer and sled. 


Test and Evaluation. The contractor shall conduct a test and evaluation program to verify 


that the vehicle, including contractor furnished equipment and Government furnished 


equipment, performs in accordance with the requirements of Specification NO-794-78-003. 


Conduct development tests of selected items as required to evaluate performance 
and reliability. 

Prepare a factory acceptance test plan covering factory-level tests and special tests 
for Navy approval (CDRL Sequence A00A). 

Conduct factory acceptance tests in accordance with the approved test plan, 
analyze test data, and prepare test reports (CDRL Sequence AOOB). 

Support special tests at the Ocean Simulation Facility, Panama City, FL, analyze 


test data, and prepare special report (CDRL A00C). 


System Engineering. This task covers the integrating activities related to applied engineer- 


ing disciplines. This activity will: 


e Coordinate with NAVSEA and GFE suppliers to define requirements and 


establish physical and functional interfaces between the vehicle, support 


equipment, and the related GFE. 


* Maintain coordination with NAVSEA to develop software interfaces for the GFE 


IMU to assure successful operation of the CSTV. 
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Maintain positive interface control between the vehicle and support equipment. 
Conduct internal design reviews. 

Prepare the Critical Design Review Agenda (CDRL Sequence A003), CDR Data 
(CDRL Sequence A002), and conduct the CDR. 

Review Test Plans and Test Reports. 

Prepare for Navy approval, a Quality Assurance Plan (CDRL Sequence AOOE) 
and conduct a quality assurance program in accordance therewith. 

Conduct and monitor Reliability and Maintainability programs. 

Conduct and monitor Systems Safety and Human Engineering programs. 
Prepare requests for approval of nonstandard parts (CDRL Sequence A007). 
Prepare Quarterly Technical Progress Reports (CDRL Sequence A009) and a 
Final Engineering Report. 

Construct full scale soft mock-ups of the test vehicle and support equipment, as 


required. 


Systems Management. The contractor shall provide and maintain an organization structure 


to ensure effective direction and management that will: 


Formulate decisions and exercise technical management. 

Establish rational priorities and clearly defined responsibilities within each 
discipline. 

Monitor and evaluate activities of all disciplines to ensure feedback of complete 
and accurate information for in-depth program visibility. 

Establish and maintain a program master schedule encompassing all tasks 


necessary for program accomplishment. 
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Negotiate resource allocations with all organizations, document results, and 
release authorizing documents for resource utilization. 

Coordinate the evaluation and selection of qualified subcontractors and suppliers. 
Administer all subcontract and purchase effort. 

Prepare Monthly Letter Status Reports (CDRL Sequence A001). 

Ensure thorough evaluation and proper authorization of all design changes. 
Confirm the incorporation of all authorized changes into hardware. 

Prepare Engineering Change Proposals, Deviations, and Waivers (CDRL 
Sequence A008). 

Ensure the proper preparation and timely submission of all data specified by the 
Contract Data Requirements List (DD Form 1423). 

Perform logistical studies and analyses. 

Prepare a list of recommended spare parts. 

Prepare a list of special and general purpose electronic test equipment (CDRL 
Sequence A005). 

Prepare a Technical Maintenance Manual, Operator's Manual and Parts List 
(CDRL Sequence A006). 

Perform liaison with the government and GFE suppliers to ensure integration of 
GFE Operation and Maintenance Manuals. 

Inspect and test GFE as received to ensure proper operation before installation. 
Provide protection and control of GFE to prevent damage during handling and 
storage. 


Provide field support as directed by the ACO under Item 0002 of the contract. 


70 


APPENDIX B 
CONTRACT PRICING PROPOSAL (21 JUL 78) 
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